Unsupervised Model-Free Representation Learning

Numerous control and learning problems face the situation where sequences of high-dimensional highly dependent data are available, but no or little feedback is provided to the learner. To address this issue, we formulate the following problem. Given a series of observations X0, . . . , Xn coming from a large (high-dimensional) space X , find a representation function f mapping X to a finite space Y such that the series f(X0), . . . , f(Xn) preserve as much information as possible about the original time-series dependence in X0, . . . , Xn. We show that, for stationary time series, the function f can be selected as the one maximizing the time-series information h0(f(X)) − h∞(f(X)) where h0(f(X)) is the Shannon entropy of f(X0) and h∞(f(X)) is the entropy rate of the time series f(X0), . . . , f(Xn), . . . . Implications for the problem of optimal control are presented.